Notify systemd when Elasticsearch is ready#44673
Conversation
Today our systemd service defaults to a service type of simple. This means that systemd assumes Elasticsearch is ready as soon as the ExecStart (bin/elasticsearch) process is forked off. This means that the service appears ready long before it actually is, so before it is ready to receive requests. It also means that services that want to depend on Elasticsearch being ready to start can not as there is not a reliable mechanism to determine this. This commit changes the service type to notify. This requires that Elasticsearch sends a notification message via libsystemd sd_notify method. This commit does that by using JNA to invoke this native method. Additionally, we use this integration to also notify systemd when we are stopping.
jasontedor
added
>enhancement
:Delivery/Packaging
|
Pinging @elastic/es-core-infra |
alpar-t
left a comment
alpar-t
left a comment
There was a problem hiding this comment.
Would it be worth adding a packaging test to test this integration ?
|
|
||
| static { | ||
| AccessController.doPrivileged((PrivilegedAction<Object>) () -> { | ||
| Native.register(Libsystemd.class, "libsystemd.so.0"); |
There was a problem hiding this comment.
Won't this fail with the packaged distributions on OS-es that don't support systemd, or does everything in our support matrix support it ?
There was a problem hiding this comment.
We don’t try to load the native library if the environment variable ES_SD_NOTIFY is not set to “true” and in this PR we do that only in the systemd service integration (see the unit file).
I considered the existing packaging tests sufficient, since if this integration is broken due to the service type being set to I’ll look more carefully if there is deeper testing we can do but I considered the existing packaging tests sufficient. |
modules/systemd/src/main/java/org/elasticsearch/systemd/SystemdPlugin.java
Outdated
Show resolved
Hide resolved
|
@elasticmachine update branch |
rjernst
left a comment
rjernst
left a comment
There was a problem hiding this comment.
LGTM, one minor comment. Thanks for making this only included in the packages and not archives.
| //// SecurityManager impl: | ||
| //// Must have all permissions to properly perform access checks | ||
|
|
||
| grant codeBase "${codebase.jna}" { |
There was a problem hiding this comment.
Can this go below the "Very speciall jar permissions" comment please, with the rest of the jar specific grants? (yes the SM here is a jar specific grant, but the comment above is specific to secure SM)
Today our systemd service defaults to a service type of simple. This means that systemd assumes Elasticsearch is ready as soon as the ExecStart (bin/elasticsearch) process is forked off. This means that the service appears ready long before it actually is, so before it is ready to receive requests. It also means that services that want to depend on Elasticsearch being ready to start can not as there is not a reliable mechanism to determine this. This commit changes the service type to notify. This requires that Elasticsearch sends a notification message via libsystemd sd_notify method. This commit does that by using JNA to invoke this native method. Additionally, we use this integration to also notify systemd when we are stopping.
6 participants
Today our systemd service defaults to a service type of simple. This means that systemd assumes Elasticsearch is ready as soon as the ExecStart (bin/elasticsearch) process is forked off. This means that the service appears ready long before it actually is, so before it is ready to receive requests. It also means that services that want to depend on Elasticsearch being ready to start can not as there is not a reliable mechanism to determine this. This commit changes the service type to notify. This requires that Elasticsearch sends a notification message via libsystemd sd_notify method. This commit does that by using JNA to invoke this native method. Additionally, we use this integration to also notify systemd when we are stopping.